VocaRefiner: An Interactive Singing Recording System with Integration of Multiple Singing Recordings

This paper presents a singing recording system, VocaRefiner, that enables a singer to make a better singing recording by integrating multiple recordings of a song he or she has sung repeatedly. It features a function called clickable lyrics, with which the singer can click a word in the displayed lyrics to start recording from that word. Clickable lyrics facilitate efficient multiple recordings because the singer can easily and quickly repeat recordings of a phrase until satisfied. Each of the recordings is automatically aligned to the music-synchronized lyrics for comparison by using a phonetic alignment technique. Our system also features a function, called three-element decomposition, that analyzes each recording to decompose it into three essential elements: F0, power, and spectral envelope. This enables the singer to select good elements from different recordings and use them to synthesize a better recording by taking full advantage of the singer’s ability. Pitch correction and time stretching are also supported so that singers can overcome limitations in their singing skills. VocaRefiner was implemented by combining existing signal processing methods with new estimation methods for achieving high-accuracy robust F0 and group delay, which we propose to improve the synthesized quality.